Thesis Proposal: Learning Image Patch Representation for Detection, Recognition and Dynamic Foreground/Background Extraction

Using feature based (ie. interest image features that satisfy some metrics of geometric or photometric invariance [39, 50, 66, 74, 73]) or direct image methods (ie. derivatives or differences of image intensity patterns [2, 37, 49]) for 3D object/scene model construction [48, 97], image content retrieval [85, 14], object recognition [24, 66], video structure matching/parsing [83, 93, 92] and automatic mosaics generation [10] have been well exploited in computer vision community during the last decade. The advantage of feature based methods is that some geometric and photometric invariance can be encoded into the feature design and detection process. Thus features can be repeatedly detected in a relatively more stable manner with respect to image changes, illumination variations and random noises, than direct image methods. Additionally, feature based methods is usually more robust with occlusions, due to its local part-based representation. On the contrary, direct image methods will prevail when image features are hard to find or the predesigned feature detection principles are not coincident with the given vision task. Without extra efforts on designing and finding features, direct image methods can be performed very fast and often in realtime [37, 49]. As a summary, feature based methods are more likely to be employed by representing high-resolution visual scenes with many distinct ”corner” like local features; while direct image methods have more privileges by characterizing low-resolution imagery, textureless or homogenous regions, highly repeated textures and images containing dominant ”edge or ridge” like features1. In this proposal, we represent images using sets of regularly or irregularly spatially sampled rectangle subregions of interest (ROI), ie. image patches, as an intermediate solution between feature based methods and direct image method. The image patches have much lower dimensionality than a regular sized image which makes the statistical learning problem much easier. The pool of patches can be drawn randomly from larger labelled image regions, as many as what we need. Sufficient large sets of training image patches are guaranteed. Any given image can be modelled as a distribution of its sampled image patches in the feature space, which is much more flexible than direct method of modelling the image itself globally. We demonstrate its representative validity by classifying a large photo database with very diverse visual contents [68] into scene categories and segmenting nonrigid dynamic foreground/background regions in video sequences [69] with satisfying results. More precisely, we build a probabilistic discriminative model for scene recognition which is learned over thousands of labelled image patches. The trained classifier performs the photo categorization task very effectively and efficiently [68]. Breaking images into a chuck of patches enables us to build a flexible, conceptually simple and computationally efficient discriminative classifier with good